perf(gfql): memoize row-expression parser (parse_expr) by lmeyerov · Pull Request #1657 · graphistry/pygraphistry

lmeyerov · 2026-06-27T04:08:52Z

Summary

Memoizes parse_expr() — the GFQL row-expression parser — per expression string. Stacked on #1652 dev/gfql-opt-base (this PR's base); its own delta is just the three files below.

Profiling showed that once the whole-query parse is cached (the parse_cypher memo in #1652), the residual fixed per-call compile cost is dominated by parse_expr: it's invoked ~4× per query compile (each RETURN / WHERE / WITH expression) and rebuilt a Lark transformer — which defines a frozen dataclass via runtime exec — on every call.

parse_expr is a pure function of the expression text (no params, no schema — those apply later) returning a tree of @dataclass(frozen=True) nodes (scalar/tuple fields, deeply immutable), so identical expressions — re-parsed on every compile and recurring across queries (e.g. a.val > 50) — are served from an lru_cache(maxsize=1024). The non-str/empty guard stays outside the cache, so invalid input still raises every call and is never cached. (The ExprNode deep-immutability invariant this relies on is documented + drift-guarded in #1652.)

Performance (dgx-spark)

Stacked on #1652's query-parse memo, the fixed per-call cost of a repeated string Cypher query drops a further ~4.5 ms → ~1.8 ms (RETURN a @100 rows: ~6.3 → 3.6 ms) — ≈ parity with the equivalent native chain.

Safety

The returned ExprNode tree is deeply immutable (frozen dataclasses, tuple fields, zero list/dict fields), so the cached instance is safe to share by reference; nothing bypasses frozen-ness.
Only parse_expr's GFQL callers consume the result; no external mutation.
Bounded (maxsize=1024); only successful parses cached.

Tests

test_expr_parser.py: memoization identity (same string → same object), distinct expressions are distinct + cache-hit registration, invalid input not cached. Full graphistry/tests/compute/gfql/ green; ruff + mypy clean.

Stack: master ← #1652 dev/gfql-opt-base (parse_cypher memo, dtype gates, .unique() removal, the chain fast path + its correctness/observability hardening) ← #1657 (this, parse_expr memo). The base PR carries the bulk of the optimization stack and all the fast-path review hardening; this PR is the small additive parse_expr slice on top.

🤖 Generated with Claude Code

…tructured returns Squashed reconciliation of the native lazy Polars GFQL engine (was #1648's 28 commits; full history preserved at tag bak/1648) restacked onto the colleague's #1656 structured whole-entity returns + #1657 parse_expr memo. Engine: native polars hop/chain (semi/anti joins), native cypher row pipeline (select/where/order_by/group_by/unwind/projection), lazy single-hop collect-once with CPU/GPU execution targets (gfql/lazy/). NO pandas bridge — native or honest NotImplementedError (plan.md NO-CHEATING). Reconciliation with #1650 structured returns: apply_result_projection now threads `structured` to the polars path (apply_result_projection_polars). Whole-entity RETURN a flattens to {alias}.{field} columns natively (mirrors the pandas _flat_entity_field_names selection exactly), which — unlike the legacy entity-text expr — works for ANY dtype (float/temporal/nested just become columns), so polars structured == pandas structured across the board. structured=False still renders the native Cypher display string for int/string/bool single-entity nodes. _include_numeric_id_as_property is now polars-aware so id flattens identically. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…te (#1657) Code-quality pass on the parse-cache layer: - Condense the verbose dropna/dangling-edge and dup-node fast-path comments to terse one/two-liners; same for the SSOT dtype-kind note and the ExprNode immutability invariant. - Note in the SSOT comment that filter_by_dict's mirror is kept in sync by a test (not imported) to avoid an import cycle. No behavior change; mypy + ruff clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@100

Complements the Cypher whole-query parse memo in PR #1652. Once parse_cypher is cached, profiling the residual fixed per-call compile cost showed it is dominated by parse_expr() — the GFQL row-expression parser, invoked ~4x per query compile (each RETURN/WHERE/WITH expression) and, per call, rebuilding a Lark transformer that defines a frozen dataclass via runtime exec (dataclasses._process_class / _create_fn / exec churn). parse_expr() is a pure function of the expression string (no params/schema) and returns a tree of frozen dataclasses (17 frozen node types, tuple-valued fields, immutable). So identical expressions — re-parsed on every compile, and recurring across queries (e.g. `a.val > 50`) — are memoized via lru_cache(maxsize=1024). Non-str/empty guard stays outside the cache; only successful parses are cached. No source consumer outside graphistry/compute/gfql calls parse_expr, and nothing bypasses frozen-ness (no object.__setattr__). Measured (dgx-spark, median-of-9): stacked on the query-parse memo, the fixed per-call cost of a repeated string Cypher query drops a further ~4.5 ms -> ~1.8 ms (RETURN a @100 rows: ~6.3 -> 3.6 ms) — near parity with the equivalent native chain. Touches only expr_parser.py (+ its test) — disjoint from PR #1652's parser.py, so the two apply independently in either order. Tests: 3 focused expr-cache tests (memoization identity, distinct + hit registration, invalid non-caching). Full graphistry/tests/compute/gfql/: 2512 passed, 16 skipped, 15 xfailed; only 2 unrelated in-container artifacts fail (networkx setup.py packaging; a cugraph "without cugraph" shortest-path test in an image that has cugraph). ruff + mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

lmeyerov changed the title ~~perf(gfql): memoize Cypher parse per query string~~ perf(gfql): memoize Cypher + row-expression parsers Jun 27, 2026

lmeyerov force-pushed the dev/gfql-cypher-parse-cache branch from d48e01c to b26f27f Compare June 27, 2026 04:21

lmeyerov changed the title ~~perf(gfql): memoize Cypher + row-expression parsers~~ perf(gfql): memoize row-expression parser (parse_expr) Jun 27, 2026

lmeyerov force-pushed the dev/gfql-cypher-parse-cache branch 5 times, most recently from 56b5730 to 19e0751 Compare June 28, 2026 20:00

lmeyerov force-pushed the dev/gfql-cypher-parse-cache branch from a483626 to cb8039c Compare June 28, 2026 21:02

lmeyerov changed the title ~~perf(gfql): memoize row-expression parser (parse_expr)~~ perf(gfql): generic engine optimization stack — parse memoization, dtype gates, chain fast path Jun 28, 2026

lmeyerov force-pushed the dev/gfql-cypher-parse-cache branch from 9b3fca8 to 47b7a43 Compare June 28, 2026 23:19

lmeyerov force-pushed the dev/gfql-cypher-parse-cache branch from 47b7a43 to a284bdb Compare June 29, 2026 00:16

lmeyerov changed the base branch from master to dev/gfql-opt-base June 29, 2026 00:54

lmeyerov force-pushed the dev/gfql-cypher-parse-cache branch from a284bdb to 6809f39 Compare June 29, 2026 00:59

lmeyerov changed the title ~~perf(gfql): generic engine optimization stack — parse memoization, dtype gates, chain fast path~~ perf(gfql): memoize row-expression parser (parse_expr) Jun 29, 2026

lmeyerov mentioned this pull request Jun 29, 2026

perf(gfql): general optimizations base (parse memoization + temporal dtype-gate) [PR0] #1652

Merged

lmeyerov force-pushed the dev/gfql-cypher-parse-cache branch from 6809f39 to cb821ce Compare June 29, 2026 01:29

lmeyerov deleted the branch master June 29, 2026 02:08

lmeyerov closed this Jun 29, 2026

lmeyerov reopened this Jun 29, 2026

lmeyerov changed the base branch from dev/gfql-opt-base to master June 29, 2026 02:10

lmeyerov merged commit c6ca4fe into master Jun 29, 2026
88 of 89 checks passed

lmeyerov deleted the dev/gfql-cypher-parse-cache branch June 29, 2026 02:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf(gfql): memoize row-expression parser (parse_expr)#1657

perf(gfql): memoize row-expression parser (parse_expr)#1657
lmeyerov merged 1 commit into
masterfrom
dev/gfql-cypher-parse-cache

lmeyerov commented Jun 27, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

lmeyerov commented Jun 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Performance (dgx-spark)

Safety

Tests

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

lmeyerov commented Jun 27, 2026 •

edited

Loading